Systems and methods for acoustic control of a vehicle&#39;s interior

ABSTRACT

Systems, methods, and computer-readable media are disclosed for acoustic control of a vehicle&#39;s interior. Example methods may include detecting environmental sounds external to a cabin of a vehicle and determining at least one first sound from the environmental sounds, wherein the cabin is configured to reduce a volume of the environmental sounds below a threshold; determining location information comprising a direction and a distance of the first sound with respect to the vehicle; and generating a second sound based on the first sound and the location information that reproduces a spectral feature of the first sound.

TECHNICAL FIELD

The present disclosure relates to systems and methods for acousticcontrol, and more specifically, to methods and systems for acousticcontrol of a vehicle's interior.

BACKGROUND

Today's vehicles have increasingly quieter cabins. In particular,vehicles can use sound damping materials to isolate the vehicle'sinterior from external noise. Additionally, vehicles can include noisereduction systems to help provide a quieter driving experience. Reducingcabin noise, however, may not be selective as to the type of noise thatis reduced. For example, passive or active noise reduction techniquesmay attenuate environmental sounds indiscriminately. Accordingly, soundsthat the driver or passengers may want to hear may be attenuated as muchas sounds that the occupants may not want to hear. Therefore, what areneeded are systems and methods for providing acoustic control of avehicle's interior.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a diagram of an example environmental context for providingacoustic control of a vehicle's interior, in accordance with exampleembodiments of the disclosure.

FIG. 2 shows one example scenario in which acoustic control of avehicle's interior can provide a driver with increased situationalawareness, in accordance with example embodiments of the disclosure.

FIG. 3 shows a diagram of providing passengers with mapping capabilitiesfor improving a driver's situational awareness for the example scenariodescribed in FIG. 2, in accordance with example embodiments of thedisclosure.

FIG. 4 shows an example scenario in which the disclosed systems andmethods can perform acoustic trilateration to locate an external soundsource, in accordance with example embodiments of the disclosure.

FIG. 5 shows an example process flow describing a method for providingacoustic control of a vehicle's interior, in accordance with exampleembodiments of the disclosure.

FIG. 6 is a schematic illustration of an example server architecture forone or more servers that can be used for providing acoustic control of avehicle's interior, in accordance with one or more embodiments of thedisclosure.

DETAILED DESCRIPTION

Overview

Vehicles are increasingly acoustically isolated from the exteriorenvironment purportedly to improve a user's cabin experience. In somecases, acoustic isolation and damping has reduced the sound levelassociated with engine sounds, which may be undesired by the occupantsof the vehicle. Accordingly, a vehicle's cabin interior speakers can beused to emulate certain sounds, for example, artificial engine sounds.This sound emulation may be performed in order to provide occupants withthe acoustic feedback that they may be used to or desire to perceive. Inparticular, engine sound emulation may be implemented for electricvehicles which may generally be quieter due to (1) their use of electricmotors, and (2) the increased sound isolation provided in such vehicles'interiors as compared with fossil-fuel based vehicles.

However, drivers in such acoustically-isolated environments (forexample, electric vehicle cabins) may have reduced situationalawareness. For example, the drivers may be unable to adequately hearemergency sirens and honking sounds of other drivers that generallyserve to alert drivers to changing road environments. Moreover,conventional systems may simultaneously filter out sounds that theoccupants may find pleasing (for example, birds chirping, ocean sounds,and the like). An occupant may be required to open a window to hear suchsounds, unfortunately limiting the cooling and/or heating mechanisms ofthe vehicle. Additionally, the opening of the vehicle's windows mayresult in bothersome noise generated from high wind speeds through thevehicle cabin or generated from wind buffeting.

Today's vehicles may come equipped with a plethora of sensors. Suchsensors include, but are not limited to, microphones, cameras, internalvehicle sensors (for example, engine sensors, traction control sensors,acceleration sensors, rain sensors, and/or the like), andtelematics-based sensors (for example, 5G connectivity sensors,vehicle-to-everything (V2X) sensors, and/or the like). Vehicles are alsoincreasingly being connected to one another and to infrastructuralcomponents (for example, using V2X protocols). Accordingly, a vehiclemay use such sensors to detect, directly or indirectly, environmentalsounds such as sirens or honking sounds. In some examples, the disclosedsystems may generate and present emulated sounds to the occupants of avehicle based on the detected environmental sounds. Further, thedisclosed systems may generate sounds (1) of a given type (e.g., roadsounds, engine sounds, siren sounds, etc.) and (2) having certaincharacteristics (e.g., volume, pitch, etc.) based on the detectedsounds. In some instances, the disclosed systems may generate soundsbased on a consideration of the effects of Doppler shifting. Thegeneration of such emulated sounds may be based on user preferencesand/or safety considerations. In some examples, the generation andpresentation of emulated sounds may be performed through the use of anysuitable digital signal processing (DSP) algorithms and/or artificialintelligence (AI)-based techniques, such as deep neural networks.

In some examples, the disclosed systems may detect and extractenvironmental sounds from a single or from multiple audio channels. Forexample, the detection of the sounds may be performed using a microphonearray having a predetermined number and spatial arrangement.Furthermore, the environmental sounds may be stationary and localized inspace (for example, having a well-characterized direction and location)or may include ambient sounds that are less spatially localized. Inother cases, the environmental sounds may be associated with movingobjects having a velocity or acceleration with respect to the vehicle.Accordingly, the disclosed systems may also determine various otherproperties (such as a relative or absolute velocity vector of a soundsource and/or the vehicle) associated with the environmental sounds. Insome cases, the disclosed systems may determine a Doppler shift of suchsounds as sirens.

Depending on user preferences or original equipment manufacturer(OEM)-determined settings, the disclosed systems may use theseenvironmental sounds and their attributes to alert the occupants to theoutside world to improve the occupant's situational awareness. Forinstance, the environmental sounds may be played within a vehicle'scabin in a manner to emulate a virtually-projected sound using the cabinspeakers of the vehicle such that the perceived location of the sound issimilar to the actual real-world location of the sound. Furthermore, thedisclosed systems may (1) modify the acoustic signal to include Dopplershift effects and may (2) modify the intensity of the acoustic signalbased on user preferences as inputted via an application running on avehicle device or a user device (e.g., mobile phone).

The disclosed systems may also present visual alerts based on thedetected environmental sounds, the visual alerts presented to vehicleoccupants in addition to or in substitute for the environmental sounds.The disclosed systems may generate and present such visual alerts on avehicle's heads-up displays, center console displays, and/or the like.The disclosed systems may also present the visual alerts alongsideonboard map information showing the vehicle's current location. Forexample, the disclosed systems may present the visual alerts on avirtual map to denote the location of the sound along with other drivinginformation (for example, a vehicle's heading, speed, and/or the like).

ILLUSTRATIVE EXAMPLES

Embodiments of the present disclosure are described herein. It is to beunderstood, however, that the disclosed embodiments are merely examplesand other embodiments can take various and alternative forms. Thefigures are not necessarily to scale; some features could be exaggeratedor minimized to show details of particular components. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a representative basis forteaching one skilled in the art to variously employ the presentinvention. As those of ordinary skill in the art will understand,various features illustrated and described with reference to any one ofthe figures can be combined with features illustrated in one or moreother figures to produce embodiments that are not explicitly illustratedor described. The combinations of features illustrated providerepresentative embodiments for typical applications. Variouscombinations and modifications of the features consistent with theteachings of this disclosure, however, could be desired for particularapplications or implementations.

The disclosed systems may detect sounds external to a vehicle's cabin,determine the respective directions and locations associated withexternal sounds. Accordingly, the disclosed systems may emulate theexternal sounds within the vehicle cabin. In some examples, thedisclosed systems can emulate the directionality and location of thedetected sounds, for example, using a multi-channel and multi-zonalvehicle audio system. The disclosed systems may include systems andmethods to detect user-desired environmental sounds and filter outenvironmental sounds that are not desirable by users.

In some examples, the disclosed systems may configure a physicalinterface and/or a human-machine interface (HMI) to accept varioususer-selected parameters. For example, the disclosed systems may use theinterface to allow users to determine desired and undesiredenvironmental sounds to be filtered out of the vehicle cabin. In someexamples, the disclosed systems may provide techniques for prioritizingthe criticality of external sounds which may be informed, at least inpart, by user input. For example, the disclosed systems may determine ahigher priority associated with sirens from emergency vehicles ascompared with user-preferred environmental sounds (for example, birdschirping). The disclosed systems may perform real-time acoustictrilateration via V2V communications with other vehicles that sampleexternal audio, as further shown and described in connection with FIG.4, below. The acoustic trilateration may be improved using map-matchingthat include the road geometry and using an onboard map database.

FIG. 1 shows a diagram of an environmental context, in accordance withexample embodiments of the disclosure. FIG. 1 represents anenvironmental context 100 that includes a vehicle 102. In some examples,the vehicle 102 may include an autonomous vehicle (AV).

In some examples the disclosed systems may include a vehicle 102 havingsensors. The sensors may include, but not be limited to, microphone(s),cameras, vehicle systems (engine, traction control, brakes,accelerometers, rain sensor, and/or the like). The vehicle 102 mayinclude an audio DSP capability that can project virtual sounds in anydirection within the cabin in combination with multiple speakers withinthe vehicle. The vehicle 102 may have various wireless-connectivitycapabilities, for example, to enable indirect measurements of variousenvironmental objects. In some examples, one wireless-connectivitycapability may include a V2X communications capability. The vehicle 102may have a location-determination capability (for example, a globalpositioning system (GPS) capability) and the ability to receive andprocess mapping information. These capabilities are discussed furtherbelow.

In some examples, the vehicle 102 may include an audio-detectioncapability. The disclosed systems may include a vehicle-basedmicrophone(s). In some examples the vehicle-based microphone(s) mayinclude exterior microphone(s). The exterior microphone may be used todetect acoustic signals of interest (e.g., bird chirping sounds 106) andextract sounds corresponding to the acoustic signal of interest from thesignals detected by the microphone(s). In other examples, the disclosedsystems may use AI-based techniques to enhance the detection andextraction of the sounds by the exterior microphone(s). For example, thedisclosed systems may use a neural network to detect audio events forsound extraction and an audio signal obtained from the environment viathe microphone. In other examples, the disclosed systems may usemultiple audio samples obtained over time or space and via themicrophone(s) to perform acoustic trilateration, as further shown anddescribed in connection with FIG. 4, below.

In some examples, the vehicle 102 may include a camera (for example, avehicle-based camera), not shown. The camera can be used to detectflashing lights and other signals associated with emergency vehicles, asdescribed further in connection with FIGS. 2-3, below. The camera canalso be used to detect other sound-emitting visual indicators in thevehicle's environment. In some examples, the vehicle may include acamera aimed internally at the driver and/or occupants that can monitorthe driver and/or occupants to improve safety and improve userexperience. In such a system, the data obtained from the interior camera(and thereafter processed using computer-vision techniques) may be usedto modify or augment the behavior of the sound extraction and filtering.This modification or augmentation can serve to improve safety, toimprove driver awareness, and to avoid driver distraction.

In other examples, the vehicle 102 may perform light-based communicationwith other vehicles or infrastructural components. For example, othervehicles may include headlights which may flash light patterns over aperiod of time. The vehicle 102 may use an optical device such as aphotodiode and/or a complementary metal-oxide-semiconductor (CMOS)-basedrolling shutter camera (not shown) to decode the light patterns that mayvary with position and time to indicate that the vehicle 102 isperforming a certain action, such as honking, or provide additionalinformation to other vehicles or infrastructural elements. The disclosedsystems may then use this information to generate emulated soundscorresponding to features of the observed action, which may be presentedto the occupants 104. In some examples, various other modifications ofthe presented sound, Doppler shift, orientation, and/or alertinformation can be performed as needed.

The disclosed systems may also use the camera to detect various roadconditions, for example the camera may be used to determine that theroads are wet or dry. The camera may be used to determine a weathercondition. For example, the camera may be used to identify blown leavesand/or rain on the windshield of the vehicle 102. The disclosed systemscan then use such information to determine that the weather is rainy.The disclosed systems may then use this information to generate emulatedsounds corresponding to features of the weather condition (e.g.,rainfall), that may be presented to the occupants. In some examples, theweather-based sounds may be modified (e.g., dampened) to suit thesensibilities of the occupants based on user-defined preferences. Forexample, the an AEB can be activated to display varying tire sounds,such as squeaking and crunching sounds that correspond with tires movingover snow, ice, water, leaves, and the like. In other examples, thedisclosed systems may be configured to be in a racing mode, and cangenerate sounds to indicate that a particular tire may be slipping,while other tires have sufficient traction.

The vehicle 102 may be equipped with a V2X communication capability. Insome examples, other vehicles in the proximity of the vehicle 102 mayprovide a direct alert of sound emission (sirens, honking horns) in theV2X transmissions to the vehicle 102. The other vehicles may detectsound emission around themselves and share such information with thevehicle. Nonlimiting examples of such information may includeinformation associated with acoustic trilateration. Such information mayuse multiple vehicles to determine localization (for example, soundorigin and/or directionality determination), as well as to determinesound intensity and Doppler shifting effects associated with thedetected sounds. In other examples, the information may be used toimprove sound extraction from the environment, and in particular, toimprove sound extraction over background noises such as road noise, windnoise, and the like. In some examples, the disclosed systems may use anysuitable security technique (e.g., encryption technique) to secure V2Xcommunications and/or detected and generated audio. For example, suchsecure communications along with detection and generated audio can beused to protect the vehicle and associated systems from situationsinvolving malicious actors. For example, such actors may transmitfraudulent audio files or V2X instructions to the detection systems ofthe vehicle. Accordingly, the disclosed systems can detect thefraudulent audio files or V2X instructions. Further, the disclosedsystems can blacklist the source of the V2X sender, for example, byblacklisting the cryptographic key associated with the V2X sender.

As noted, the vehicle 102 may include various vehicle-based sensors.Nonlimiting examples of such sensors may include a wiper rain sensor anda traction control sensor. The disclosed systems may use the rain sensorto detect rain above some threshold and play a rain alert or ambientsounds within the vehicle's cabin. The disclosed systems may also usethe traction control sensors to determine a loss of traction when thevehicle is traveling on icy terrain, is on a hydroplane, and/or the likeand generate an audio-based alert within the cabin. In some examples,the disclosed systems may use an ADAS and/or an AV application togenerate descriptions of what the vehicle senses and explanations as towhy the disclosed systems perform certain actions (e.g. slow down due torain). The disclosed systems can present the descriptions and/orexplanations to users via a display or via audio, and may store thedescriptions and/or explanation in storage for audit purposes.

The vehicle 102 may be in communication with various navigation ormapping databases. The disclosed systems may use such databases toobtain information associated with the area in which the vehicle istraveling. For example, the disclosed systems may use the databases todetermine the noise floor level of an urban environment versus a ruralenvironment. The disclosed systems may then filter out ambient orbackground sounds based on the determined noise floor level in theenvironment in which the vehicle is navigating. In other examples, thedisclosed systems may use the databases to determine the presence oftrain tracks, underpasses, overpasses, wildlife areas, oceanicenvironments, and/or the like. Accordingly, the databases may serve tohelp the vehicle 102 both navigating the environment and determiningsound information associated with the environment. For example, thedisclosed systems may use the map data to determine sounds associatedwith trains coming on the train tracks and thereby notify the passengersof the vehicle 102 (e.g., via an audio alert played in the vehicle'scabin) about the possibility of a train crossing the vehicle's path. Inother circumstances, a train may travel upon an overpass bridge.Further, the overpass bridge may be identified based on map information.The disclosed systems may determine that there can be no vehicle andtrain interaction due to the height difference between the routes of thetrain and the vehicle. In such a case, the vehicle may determine not topresent audio or related information to the driver/passengers in thevehicle and thus, to not include train sounds in the presented audio tothe driver and/or passengers of the vehicle.

The vehicle 102 may have various wireless connectivity and/or Internetof things (IoT) capabilities to determine various environmental dataand/or information, which may serve in customizing the vehicles internalacoustic environment. Nonlimiting examples of such data and/orinformation may include whether information such as wind, rain, snowinformation and/or the like. For example, the vehicle 102 maycommunicate with various IOT devices disposed on various infrastructuralcomponents of the environment in which the vehicle is traveling andobtain any relevant information about the acoustic nature of theenvironment. This information may be used to generate sounds in thevehicle's interior emulating such an acoustic environment.

The disclosed systems may include processors that are capable ofexecuting operations based on logic and algorithms. In some examples,the disclosed systems may perform sensor analysis. In particular, thedisclosed systems may perform sensor analysis to determine a noise floorbaseline based on sampled real-time sensor data. In some examples, thedisclosed systems can detect the presence of an ongoing conversation orphone call in the vehicle. Accordingly, the disclosed systems maydetermine to not present certain sound types (e.g., ocean noises, birdchirping noises, etc.) while such conversations or phone calls areongoing. The disclosed systems may further perform a comparison to areferenced noise-floor baseline using crowd-sourced geolocation data orinformation on an external database. This crowd-sourced geolocation dataand/or information may be obtained from a cloud-based database (forexample, a public cloud database, a private cloud database, and/or ahybrid cloud database).

The vehicle 102 may include systems that may perform sensor fusion ofimages and audio obtained by the vehicle 102 and/or any associateddevices to determine sound types, directionality, distance, and/or thelike. For example, the systems may perform acoustic trilateration. Insome examples, the acoustic trilateration may be performed using threeor more microphones. The microphones may be present on one vehicle ormay be present on separate vehicles. In other examples, the microphonesmay be distributed among different vehicles which can communicatedetected sounds using V2X communications. The acoustic trilateration mayallow for sound source identification and acoustic signal extraction. Inother examples, the disclosed systems may categorize sound types toalert users of emergent environmental situations. The disclosed systemsmay also enable users to modify preference settings reflecting desiredand undesired sounds via an HMI menu 103. The disclosed systems mayprovide alerts, such as alarm sounds in certain critical conditions suchas potential collision conditions. The disclosed systems may allow auser to control ambient sound presentation preferences, such as apreferred background noise level. The disclosed systems may also allowthe user to specify desired and undesired sounds. As noted, the vehicle102 may enable a variety of audio and alerts to be presented to users.For example, the vehicle 102 may enable audio alerts, ambient acousticnoises, visual alerts, haptic feedback, visual indications of thelocation of vehicles (for example on a HUD or center console display ofa vehicle), and/or the like.

The vehicle 102 may include any suitable vehicle such as a motorcycle,car, truck, recreational vehicle, etc., and may be equipped withsuitable hardware and software that enables it to communicate over anetwork, such as a local area network (LAN).

As noted, the vehicle 102 may include a variety of sensors that may aidthe vehicle in determining an environment in which the vehicle is in.The sensors may include RADAR, LIDAR, cameras, magnetometers,ultrasound, barometers, and the like (to be described below). In oneembodiment, the sensors and other devices of the vehicle 102 maycommunicate over one or more network connections. Examples of suitablenetwork connections include a controller area network (CAN), amedia-oriented system transfer (MOST), a local interconnection network(LIN), a cellular network, a Wi-Fi network, and other appropriateconnections such as those that conform with known standards andspecifications (e.g., one or more Institute of Electrical andElectronics Engineers (IEEE) standards, and the like).

As noted, the vehicles 102 may include various location-determinationdevices in addition to satellite-based location-determination devices.These devices may be used to identify the location of the vehicle, trackthe vehicle on a map (e.g., an HD map), track other vehicles inproximity to the vehicle, provide updates on the location of a givenvehicle to other vehicles, and generally support the operationsdescribed herein. For example, the vehicle 102 may include magneticpositioning devices such as magnetometers, which may offer an indoorlocation determination capability. Magnetic positioning may be based onthe iron inside buildings that create local variations in the Earth'smagnetic field. Un-optimized compass chips inside devices in the vehicle102 may sense and record these magnetic variations to map indoorlocations. In one embodiment, the magnetic positioning devices may beused to determine the elevation of the vehicle 102. Alternatively oradditionally, a barometer device may be used to determine the elevationof the vehicle 102. In another embodiment, barometers and pressurealtimeters may be a part of the vehicle and may measure pressure changescaused by a change in altitude of the vehicles 102.

In one embodiment, the vehicle 102 may use one or more inertialmeasurement devices (not shown) to determine the respective vehicles'position in order to track the vehicles and/or to determine the locationof various sound sources in the vehicle's environment with respect to amap (e.g., an HD map). The vehicles 102 may use dead reckoning and otherapproaches for positioning of the vehicle using an inertial measurementunit carried by the vehicles 102 sometimes referring to maps or otheradditional sensors to constrain the inherent sensor drift encounteredwith inertial navigation. In one embodiment, one or moremicroelectromechanical systems (MEMS) based inertial sensors may be usedin the inertial measurement unit of the vehicle; however, the MEMSsensors may be affected by internal noises which may result in cubicallygrowing position error with time. In one embodiment, to reduce the errorgrowth in such devices, a Kalman filtering based approach may be used,by implementing software algorithms on software modules associated withthe various devices in the vehicle 102.

In one embodiment, the inertial measurements may cover one or moredifferentials of motion of the vehicle 102, and therefore, the locationmay be determined by performing integration functions in the softwaremodules, and accordingly, may require integration constants to provideresults. Further, the position estimation for the vehicle 102 may bedetermined as the maximum of a two-dimensional or a three-dimensionalprobability distribution which may be recomputed at any time step,taking into account the noise model of all the sensors and devicesinvolved. Based on the vehicles' motion, the inertial measurementdevices may be able to estimate the vehicles' locations by one or moreartificial intelligence algorithms, for example, one or more machinelearning algorithms (e.g., convolutional neural networks). The disclosedsystems may use any of the devices mentioned above in combination withthe location-determination signals to determine the location of thevehicle, determine the location of other vehicles, and/or determine thelocation of various sound sources in the vehicle's environment.

In some examples, the disclosed systems can use an indoor positioningsystem (IPS) in connection with certain infrastructural components todetermine the location of the sound sources and/or vehicles withincreased accuracy. Further, the IPS may be used to determine thelocation of the vehicle on a map (e.g., an HD map), for example, inlocations where satellite navigation signals are inadequate. Inparticular, an IPS may refer to a system to locate objects (e.g., thevehicle 102) inside a building such as a parking structure using lights,radio waves, magnetic fields, acoustic signals, or other sensoryinformation collected by mobile devices (e.g., user devices or vehicledevices). IPS's may use different technologies, including distancemeasurement to nearby anchor nodes (nodes with known fixed positions,e.g. Wi-Fi and/or Li-Fi access points or Bluetooth beacons, magneticpositioning, and/or dead reckoning). Such IPSs may actively locatemobile devices and tags or provide ambient location or environmentalcontext for devices to get sensed. In one embodiment, an IPS system maydetermine at least three independent measurements to unambiguously finda location of a particular vehicle 102 or a sound source.

In some examples, the vehicle antennas may include any suitablecommunications antenna. Some non-limiting examples of suitablecommunications antennas include Wi-Fi antennas, Institute of Electricaland Electronics Engineers (IEEE) 802.11 family of standards compatibleantennas, directional antennas, non-directional antennas, dipoleantennas, folded dipole antennas, patch antennas, multiple-inputmultiple-output (MIMO) antennas, or the like. The communications antennamay be communicatively coupled to a radio component to transmit and/orreceive signals, such as communications signals to and/or from thevehicles. For example, the disclosed systems may transmit signals toother vehicles to inform the other vehicles to take at least one action(e.g., brake, accelerate, make a turn, and/or the like) based on adetermination of the state of the sound sources.

In some examples, the vehicle 102 may have on-board units (not shown)may include microcontrollers and devices that can communicate with eachother in applications without a host computer. The on-board unit may usea message-based protocol to perform internal communications. Further,the on-board unit can cause a transceiver to send and receive messages(for example, vehicle-to-everything, V2X, messages) to and frominfrastructural components and to other vehicles' on-board units.

Further, various devices of the vehicle 102 and/or infrastructuralcomponents (e.g., smart traffic signals, roadside units, IPS systems,and/or the like) may include any suitable radio and/or transceiver fortransmitting and/or receiving radio frequency (RF) signals in thebandwidth and/or channels corresponding to the communications protocolsutilized by any of the vehicle devices to communicate with each otherand/or with infrastructural components. The radio components may includehardware and/or software to modulate and/or demodulate communicationssignals according to pre-established transmission protocols. The radiocomponents may further have hardware and/or software instructions tocommunicate via one or more Wi-Fi and/or Wi-Fi direct protocols, asstandardized by the Institute of Electrical and Electronics Engineers(IEEE) 802.11 standards. In certain example embodiments, the radiocomponent, in cooperation with the communications antennas, may beconfigured to communicate via 2.4 GHz channels (e.g. 802.11b, 802.11g,802.11n), 5 GHz channels (e.g. 802.11n, 802.11ac), or 60 GHZ channels(e.g. 802.11ad). In some embodiments, non-Wi-Fi protocols may be usedfor communications between devices, such as Bluetooth, dedicatedshort-range communication (DSRC), Ultra-High Frequency (UHF) (e.g. IEEE802.11af, IEEE 802.22), white band frequency (e.g., white spaces), orother packetized radio communications. The radio component may includeany known receiver and baseband suitable for communicating via thecommunications protocols. The radio component may further include a lownoise amplifier (LNA), additional signal amplifiers, ananalog-to-digital (A/D) converter, one or more buffers, and digitalbaseband.

Typically, when an example vehicle 102 establishes communication withanother vehicle (not shown) and/or establishes communication with ainfrastructural component device, the vehicle 102 may communicate in thedownlink direction by sending data frames (e.g. a data frame which cancomprise various fields such as a frame control field, a duration field,an address field, a data field, and a checksum field). The data framesmay be preceded by one or more preambles that may be part of one or moreheaders. These preambles may be used to allow the user device to detecta new incoming data frame from the vehicle device. A preamble may be asignal used in network communications to synchronize transmission timingbetween two or more devices (e.g., between the vehicle 102 device andinfrastructural component device and/or between the devices of separatevehicles). As noted, the data frames may be used to transmit informationbetween vehicles so that a given vehicle may perform at least one actionas a result of the detection of an on or off state of a sound source orthe determined location of other vehicles (e.g., based ontrilateration). Nonlimiting examples of such actions include breaking,turning, accelerating, turning on hazards, and/or the like.

In another aspect, the environmental context 100 may include one or moresatellites 130 and one or more cellular towers 132. The satellites 130and/or the cellular towers 132 may be used to obtain locationinformation and/or to obtain information from various databases such asdatabases having HD maps. In other aspects, the disclosed systems maytransmit information associated with sound sources and/or vehicles(e.g., the sound sources' respective locations, the sound sources'respective states, and/or the like). As noted, the vehicle 102 may havetransceivers, which may in turn include one or more location receivers(e.g., global navigation satellite system (GNSS) receivers) that mayreceive location signals (e.g., GNSS signals) from one or moresatellites 130. In another embodiment, a receiver may refer to a devicethat can receive information from satellites (e.g., satellites 130) andcalculate the vehicles' geographical position.

In some examples, the vehicles (e.g., such as vehicle 102) may beconfigured to communicate using a network, wirelessly or wired. Asnoted, the communications may be performed between vehicles, forexample, to inform a given vehicle to an action to take based on thestate of a sound source. The network may include, but not limited to,any one of a combination of different types of suitable communicationsnetworks such as, for example, broadcasting networks, public networks(for example, the Internet), private networks, wireless networks,cellular networks, or any other suitable private and/or public networks.Further, any of the communications networks may have any suitablecommunication range associated therewith and may include, for example,global networks (for example, the Internet), metropolitan area networks(MANs), wide area networks (WANs), local area networks (LANs), orpersonal area networks (PANs). In addition, any of the communicationsnetworks may include any type of medium over which network traffic maybe carried including, but not limited to, coaxial cable, twisted-pairwire, optical fiber, a hybrid fiber coaxial (HFC) medium, microwaveterrestrial transceivers, radio frequency communication mediums, whitespace communication mediums, ultra-high frequency communication mediums,satellite communication mediums, or any combination thereof.

In some examples, the disclosed systems may analyze map informationassociated with an environment of the vehicles, previous vehiclelocations in a given environment, sound source locations,infrastructural updates regarding the transportation network (forexample, sound sources due to construction or other activities) and/orthe like. The database may be controlled by any suitable system,including a database management system (DBMS), discussed further inconnection with FIG. 6, below. The DBMS may use any of a variety ofdatabase models (for example, relational model, object model, etc.) andmay support any of a variety of query languages to obtain informationfrom the database. In some examples, the database may include acloud-based database or a vehicle-based database.

Certain embodiments of the disclosure are now described in the contextof an example non-limiting scenario. The scenario can include userstraveling in a luxury vehicle, the users having preferences thatindicate a fondness for sounds associated with nature. The users mayprovide input to the disclosed systems indicative of a preference for agenerally quiet vehicle cabin. However, the users may also indicate adesire to hear certain environmental sounds, such as birds chirpingalong rural drives.

In this example scenario, the disclosed systems (e.g., the vehicleand/or vehicle-based devices) may detect a base noise floor associatedwith the exterior environment. The disclosed systems may reduce externalunwanted (for example, road noise, engine noise, traffic, and the like)using noise cancellation techniques. In some cases, the disclosedsystems may perform noise suppression of sounds that are disliked basedon previously indicated user preferences. The disclosed systems mayperform the noise reduction using a user selection of predeterminedsound-level preferences or a user-customized sound-level preference. Inother examples, the system may learn a user's sound preferences usingfeedback from the user and the use of a machine learning algorithm suchas a Gaussian process. The disclosed systems may use a vehicle'sinterior and exterior microphones along with AI-based techniques (forexample, machine learning and/or deep learning techniques) to allow nearreal-time acoustic signature identification of external sounds. Forexample, the disclosed systems may use such techniques to identify themost common birds and their associated chirps for the region the vehicleis currently located in. For example, the disclosed systems can look upthe birds and associated chirps from an external database accessible viathe vehicle's communication systems and using associated vehicleantennae. The location may be determined using communication with anexternal database and/or via GPS receivers. The disclosed systems maythen play emulated chirp sounds matching the identified chirps of suchbirds in the vehicle's cabin.

In some examples, the disclosed systems may use a vehicle's cameras anduse AI-based techniques (for example, machine learning and/or deeplearning techniques) to allow near real time visual identification ofcertain external objects. In some examples, the disclosed systems canuse the visual identification to present information associated with theobjects to the users. For example, the disclosed systems may allow forthe visual identification of the most common birds in the region thevehicle is currently navigating. The disclosed systems may obtaininformation associated with the birds from external databases and maythen present information associated with such birds along with picturesand/or videos of the birds to the vehicle's occupants. In some examples,the information may be presented on any suitable display of the vehicle.Alternatively or additionally, the information may be presented viaaudio at the vehicle's speakers, for example, using a text-to-speechengine. In other examples, the disclosed systems may use the vehicle'sinfotainment system for processing and filtering the external sounds,thereby allowing the near real time playback of a portion of the sounds(e.g., targeted bird chirps).

Further embodiments of the disclosure are now described in the contextof another example non-limiting scenario. This scenario can include aluxury vehicle driving on a scenic coastal drive. Further, the occupantsof the luxury vehicle may prefer a generally quiet vehicle cabin.However, the occupants may prefer to hear the ocean's natural soundsduring their drive.

In this example, the disclosed systems may detect a base noise floorassociated with the exterior environment. The disclosed systems mayreduce external unwanted (for example, road noise, engine noise,traffic, and the like) using noise cancellation techniques. Thedisclosed systems may perform the noise reduction using a user selectionof predetermined sound-level preferences or a user-customizedsound-level preference.

Continuing with this example, the disclosed systems may use navigationand mapping data to determine the vehicle's current location withrespect to oceans and/or waterways. For instance, the disclosed systemsmay determine the distance between the vehicle's location and the oceanand/or waterway and calculate the effect of the distance on the sounddecibel level. The disclosed systems may similarly identify wind andweather information to determine the external environment's likelyimpact on real-world audio feedback. The disclosed systems may also usethe information to determine the relationship between a sound decibellevel of ocean and/or waterway versus the distance from ocean and/orwaterway.

In another example, a driver may be an accomplished classically trainedviolinist (e.g. named “Meira”). The disclosed systems may learn (e.g.,via machine learning) the driver's preferences such that the disclosedsystems filter out external music disliked by the driver (e.g. vocalmedieval chants), and present extracted sounds enjoyed by the driver(e.g., Shostakovich symphony No. 5 in D minor, Prokofiev violin concertoNo. 1 in D major, Schubert string quartet No. 14 in D minor, Sibeliusviolin concerto in D minor, etc.) when the user drives in proximity tosuch sounds.

The disclosed systems may use a vehicle's interior and exteriormicrophones along with AI-based techniques (for example, machinelearning and/or deep learning techniques) to allow near real-timeacoustic signature identification of external sounds. For example, thedisclosed systems may use such techniques to identify typical wave andocean current sounds. The disclosed systems may then play emulated waveand ocean current sounds matching the identified typical wave and oceancurrent sounds in the vehicle's cabin. The disclosed systems may use avehicle's cameras and use AI-based techniques to allow near real timevisual identification of visible coastal regions, as well as the status,spacing, and order of ocean waves to allow appropriate synchronizationof audio processing. The disclosed systems may use the vehicle'sinfotainment system processing and filtering capabilities to allow forthe near real-time playback of targeted wave and ocean current sounds.

In certain aspects, the disclosed systems may operate with a vehicledesignated to be in a race mode or sport mode of operation. In thesesituations, the disclosed systems may reduce or remove road or enginenoise on the way to the track, but allow all road and engine noise to beheard when at the track or amplify other engine noise from othervehicles to enhance the auditory experience of the driver or passengers.In some examples, the disclosed systems may further enhance the roadnoise as compared to a traditional vehicle. For example, a wheel may betaken to its traction limit such that slip may occur to generatesquealing noises. This squeal sound may propagate through the vehicle'sbody such that determining which individual wheel has lost traction maybe difficult to determine relative to the noise floor and otherdampening effects. The disclosed embodiments can enable the vehicle todetect wheel slip from a vehicle control module and via audio detectionof an individual wheel. Further, the disclosed systems can present animproved audio signal (e.g., denoised or artificial audio signal) toimprove a driver's situational awareness.

In other aspects, the disclosed systems may provide assistance for thedeaf and individuals with hearing loss. For example, the disclosedsystems may alert those that cannot hear exterior sounds with a visualalert of important road sounds. For examples, a deaf occupant may bealerted to an emergency vehicle's position, speed, and proximity througha visual alert.

Another embodiment of the disclosure is now described in the context ofanother example non-limiting scenario. FIG. 2 shows one example scenarioin which acoustic control of a vehicle's interior can provide a driverwith increased situational awareness, in accordance with exampleembodiments of the disclosure. This scenario can include a vehicle 202in proximity to an active emergency vehicle 206 on an intersection 208.There may be other vehicles (e.g., vehicle 204) also in proximity to thevehicle 202. In this case, the vehicle's occupants may prefer a quietvehicle cabin. However, the vehicle's occupants may also need to be madeaware of the presence, heading, and/or speed of any nearby activeemergency vehicles such as active emergency vehicle 206.

In this case, the disclosed systems may detect a base noise floorassociated with the exterior environment. The disclosed systems mayreduce external unwanted (for example, road noise, engine noise,traffic, and the like) using noise cancellation techniques. Thedisclosed systems may again perform the noise reduction using a userselection of predetermined sound level preferences or a user customizedsound level preferences.

In some embodiments, the vehicle 202 may use any suitable V2V protocolto detect nearby emergency vehicles (e.g., active emergency vehicle 206)within a predetermined radius of the vehicle 202. The disclosed systemsmay use a vehicle's 202 interior and exterior microphones along withAI-based techniques to allow near real-time acoustic signatureidentification of external sounds. For example, the disclosed systemsmay use such techniques to identify typical emergency vehicle 206 soundsand siren patterns.

The disclosed systems may use a vehicle's 202 cameras and use AI-basedtechniques to allow near real time visual identification of certainexternal objects such as the active emergency vehicle 206. In someexamples, the disclosed systems can use the visual identification topresent information associated with the objects to the users. Forexample, the disclosed systems may allow for the visual identificationvisible emergency vehicle lights and signature color and timing patternsassociated with the active emergency vehicle 206. The disclosed systemsmay then present information associated with the active emergencyvehicle 206 along with pictures and/or videos of the emergency vehicle206 to the vehicle's 202 occupants. In some examples, the informationmay be presented on any suitable display of the vehicle 202.

In some examples, the disclosed systems may also use the vehicle's 202cameras to determine the distance and path of visible emergency vehiclelighting associated with the active emergency vehicle 206. In otherexamples, the disclosed systems may use the vehicle's 202 infotainmentsystem for processing and filtering the external sounds, therebyallowing the near real time playback of the active emergency vehicle's206 siren. The disclosed systems can play the emulated emergency sirensto include the directionality of the siren as well as include Dopplereffects for increasing the users' perception of the realism andintuitiveness of the siren sounds location. Along with the playback ofthe active emergency vehicle's 206 siren, the vehicle's 202 interior HMIcould also indicate the direction and path of the identified activeemergency vehicle 206 for improved safety and awareness, as shown anddescribed in connection with FIG. 3, below. In particular, a heads-updisplay or infotainment display associated with the vehicle 202 can beused for the display of the active emergency vehicle's 206 location andto provide warning notifications.

As noted, the disclosed systems may enable acoustic trilateration ofexternal sound sources. In some examples, such external sound sourcesmay not have sufficient corresponding visual cues. For example, theexternal sound sources may include a honking horn, a whistle from policeofficer directing traffic, and/or the like. The disclosed systems mayuse acoustic trilateration to facilitate vision-based identification ofsound sources. In some examples, a particular form of trilateration canbe used with acoustic sampling of the external environment to attempt tonarrow down the direction from where the sound originates. Suchtrilateration may be useful for identifying the location of sounds fromvehicles that cannot communicate via V2V protocols. Such vehicles (forexample, older emergency vehicles) may not be able to properly identifythemselves. However, such vehicles may be occluded or lack visual cuesto identify them visually by a driver or by a vehicle camera.

Accordingly, the disclosed systems can identify such sound sources usingvehicle-based microphones. In particular, the disclosed systems may usethe microphones to sample external sounds. Alternatively oradditionally, the disclosed systems may receive sampled and normalizedexternal sounds using an external device and/or via a V2V communication.The disclosed systems can process the external sound, characterize thesound via its acoustic signature, and analyze the external sound toestimate the approximate sound pressure level as sampled. Further, thedisclosed systems may broadcast this information to any otherV2V-capable vehicles in proximity.

In some examples, if at least three messages of this type are receivedby a V2V capable vehicle, the approximate location of the originationpoint of the sound can be determined. The disclosed systems may performsuch a determination using trilateration techniques and a map database.The map database may include sounds characterized as being most likelyto have emanated from a vehicle (e.g. honking horn, screeching tires,and/or the like).

In an example scenario, an acoustic-trilateration capable vehicle canreceive a V2V message from two other vehicles broadcasting messageshaving a sound pressure level of a corresponding sampled sound and thesampled sound's characterization type. The receiving vehicle can pairthe V2V information with its own sampling and characterizationinformation to determine where the location of the sound originatedfrom. In particular, because the receiving vehicle knows its ownlocation, as well as the location of the two other vehicles, the vehiclecan use trilateration techniques to make the location determination.Further, in the case of vehicle-emanated sounds, the disclosed systemsmay further incorporate relevant knowledge of road geometry (forexample, as determined from an external database) to make the locationdetermination.

FIG. 3 shows a diagram 300 of providing passengers with mappingcapabilities for improving a driver's situational awareness for theexample scenario described in FIG. 2, in accordance with exampleembodiments of the disclosure. As noted, the disclosed systems may alsopresent visual alerts based on the detected environmental sounds, thevisual alerts presented to vehicle occupants in addition to or insubstitute for the environmental sounds. The disclosed systems maygenerate and present such visual alerts 306 on a vehicle's heads-updisplays, center console displays, and/or the like. The disclosedsystems may also present the visual alerts alongside onboard mapinformation 302 showing the vehicle's current location. For example, thedisclosed systems may present the visual alerts on a virtual map todenote the location of the sound along with other driving information(for example, a vehicle's heading, speed, and/or the like).

FIG. 4 shows an example scenario in which the disclosed systems andmethods can perform acoustic trilateration to locate an external soundsource, in accordance with example embodiments of the disclosure.Diagram 400 shows four vehicles (vehicles 404, 406, 408, and 410) at anintersection 401, with three of the vehicles (vehicles 406, 408, and410) being V2V capable, and one vehicle (vehicle 404) which is not V2Vcapable. As described below, the vehicle 404, which may not be V2Vcapable, can still be identified as the origination of the honk sound.Any of the V2V-capable vehicles can broadcast their own messagesidentifying themselves as the vehicle which is honking. However, withacoustic trilateration, V2V-capable vehicles can collectively sample theaudio, characterize, and broadcast the sampled level to each other forany of them to make their own determination of the location of the audiosource.

The location-determination process can include processing thebroadcasted data and GPS locations of the three samples to producepossible ranges of where the origination point of the sound may reside.This can be represented by circles 403, 405, and 407 of FIG. 4, whichcan represent areas in which audio can be sampled by each respectivevehicles' external microphone. The vehicle 410 on the south side of theintersection can sample the sound, characterize sound as a vehicle hornvia its acoustic signature, and can determine that sound was sampled ata given amplitude (for example, about 65 dB).

Continuing with this example, the vehicle 406 on the west side of theintersection 401 can perform the same procedure to determine that thesound has a different amplitude, for example, an amplitude of about 70dB. The vehicle 408 on the east side of the intersection 401 can takethe data from the two other vehicles and combines the data with its ownsample of about 60 dB to determine that the honking vehicle 404 may beon the west side of the intersection 401. The clustering of intersectionpoints 412 may be approximately off to the side of the road, but theknowledge of the road geometry via an on-board map database can resultin the east-side vehicle 408 being able to determine (via mapidentification) that the honking vehicle 404 is most likely the vehicledirectly behind the V2V-capable vehicle 406 on the west side of theintersection 401. For vehicles with multiple microphones separated oneither side of a given sampling vehicle, the distance ranges and soundssource orientation can be even more precise and less prone to error.However, a single microphone can still properly determine the sourcelocation with data from at least two other vehicles as depicted indiagram 400.

As noted, any one of the other vehicles (vehicles 406, 408, and/or 410)that make their own determination of the honking-vehicle's 404 locationthrough acoustic trilateration may subsequently broadcast correspondinginformation to be shared with the other vehicles. The disclosed systemscan average the values of these determined locations to further increasethe accuracy and confidence of the location determination.

FIG. 5 shows an example process flow describing a method of generating acustomized acoustic environment in a vehicle, in accordance with exampleembodiments of the disclosure. At block 502, the method may includedetecting environmental sounds external to a cabin of a vehicle anddetermining at least one first sound from the environmental sounds,wherein the cabin is configured to reduce a volume of the environmentalsounds below a threshold. In some examples, determining the first soundcan include determining a plurality of sounds and respective sound typesfrom the environmental sounds, and filtering out sounds havingpredetermined sound types from the environmental sounds. Further,determining the sound types can include assigning respective prioritiesto the sounds. For example, sounds filtering out sounds further comprisefiltering out sounds that have priorities below respective thresholds.

At block 504, the method may include determining location informationincluding a direction and a distance of the first sound with respect tothe vehicle. In some examples, determining the location information mayinclude performing acoustic trilateration using V2V communications withother vehicles configured to detect the environmental sounds. Further,determining the location information may include determining a Dopplershift associated with the first sound.

At block 506, the method may include generating a second sound based onthe first sound and the location information that reproduces a spectralfeature of the first sound. In one example, spectral features caninclude frequency-based features of a signal. In particular, suchspectral features can be determined by converting the time-based signal(e.g., an audio signal) into the frequency domain using a Fouriertransform algorithm. Non-limiting examples of such spectral features caninclude a fundamental frequency, one or more frequency components, aspectral density, a spectral roll-off, and/or the like. Variouscombinations of spectral features can be used to identify the notes,pitch, timbre, bass, rhythm, melody, etc. of a particular portion of anaudio signal.

In other embodiments, generating the second sound may include reducingnoise associated with the first sound by an amount that can be based ona user preference.

At block 508, the method may include causing to play at least a portionof the second sound on a speaker of the cabin. In some examples, causingto play at least the portion of the second sound may include projectingthe second sound such that the second sound has a perceived locationthat is similar to the first sound. Further, the method may includecausing to present an image based on the second sound on a displayassociated with the cabin. In other examples, the method can includeobtaining images of an environment external to the cabin and determininga condition (e.g., emergency condition) based on an analysis of theimages. Accordingly, the disclosed systems can stop playing the portionof the second sound and can play a third sound on the speaker based onthe images.

As noted, embodiments of devices and systems (and their variouscomponents) described herein can employ AI to facilitate automating oneor more features described herein, for example, in performing sounddetection and recognition from audio captured by the microphone(s) of avehicle and performing sound localization. The components can employvarious AI-based schemes for carrying out various embodiments and/orexamples disclosed herein. To provide for or aid in the numerousdeterminations (e.g., determine, ascertain, infer, calculate, predict,prognose, estimate, derive, forecast, detect, compute) described herein,components described herein can examine the entirety or a subset of thedata to which it is granted access and can provide for reasoning aboutor determine states of the system, environment, etc. from a set ofobservations as captured via events and/or data. Determinations can beemployed to identify a specific context or action, or can generate aprobability distribution over states, for example. The determinationscan be probabilistic; that is, the computation of a probabilitydistribution over states of interest based on a consideration of dataand events. Determinations can also refer to techniques employed forcomposing higher-level events from a set of events and/or data.

Such determinations can result in the construction of new events oractions from a set of observed events and/or stored event data, whetherthe events are correlated in close temporal proximity, and whether theevents and data come from one or several event and data sources (e.g.,different sensor inputs). Components disclosed herein can employ variousclassification (explicitly trained (e.g., via training data) as well asimplicitly trained (e.g., via observing behavior, preferences,historical information, receiving extrinsic information, etc.)) schemesand/or systems (e.g., support vector machines, neural networks, expertsystems, Bayesian belief networks, fuzzy logic, data fusion engines,etc.) in connection with performing automatic and/or determined actionin connection with the claimed subject matter. Thus, classificationschemes and/or systems can be used to automatically learn and perform anumber of functions, actions, and/or determinations.

A classifier can map an input attribute vector, z=(z1, z2, z3, z4, . . ., zn), to a confidence that the input belongs to a class, as byf(z)=confidence(class). Such classification can employ a probabilisticand/or statistical-based analysis (e.g., factoring into the analysisutilities and costs) to determinate an action to be automaticallyperformed. A support vector machine (SVM) can be an example of aclassifier that can be employed. The SVM operates by finding ahyper-surface in the space of possible inputs, where the hyper-surfaceattempts to split the triggering criteria from the non-triggeringevents. Intuitively, this makes the classification correct for testingdata that is near, but not identical to training data. Other directedand undirected model classification approaches include, for example,naïve Bayes, Bayesian networks, decision trees, neural networks, fuzzylogic models, and/or probabilistic classification models providingdifferent patterns of independence can be employed. Classification asused herein also is inclusive of statistical regression that is utilizedto develop models of priority.

FIG. 6 is a schematic illustration of an example server architecture forone or more server(s) 600 in accordance with one or more embodiments ofthe disclosure. The server 600 illustrated in the example of FIG. 6 maycorrespond to a server that may be used by a vehicle (for example, anyof vehicle 102 as shown and described in connection with FIG. 1, above)on a network associated with the vehicle. In an embodiment, the server600 may include a cloud-based server that may serve to store andtransmit information (for example, HD map information including thelocation of light sources such as traffic signals, traffic information,and the like). Some or all of the individual components may be optionaland/or different in various embodiments. In some embodiments, at leastone of the servers described FIG. 6 may be located at an autonomousvehicle.

The server 600 may be in communication with an AV 640, and one or moreuser devices 650. The AV 640 may be in communication 646 with the one ormore user devices 650. Further, the server 600, the AV 640, and/or theuser devices 650 may be configured to communicate via one or morenetworks 642. The AV 640 may additionally be in wireless communicationover one or more network(s) 642 with the user devices 650 via aconnection protocol such as Bluetooth or NFC. Such network(s) 642 mayinclude, but are not limited to, any one or more different types ofcommunications networks such as, for example, cable networks, publicnetworks (for example, the Internet), private networks (for example,frame-relay networks), wireless networks, cellular networks, telephonenetworks (for example, a public switched telephone network), or anyother suitable private or public packet-switched or circuit-switchednetworks. Further, such network(s) may have any suitable communicationrange associated therewith. In addition, such network(s) may includecommunication links and associated networking devices (for example,link-layer switches, routers, etc.) for transmitting network trafficover any suitable type of medium including, but not limited to, coaxialcable, twisted-pair wire (for example, twisted-pair copper wire),optical fiber, a HFC medium, a microwave medium, a radio frequencycommunication medium, a satellite communication medium, or anycombination thereof.

In an illustrative configuration, the server 600 may include one or moreprocessors 602, one or more memory devices 604 (also referred to hereinas memory 604), one or more input/output (I/O) interface(s) 606, one ormore network interface(s) 608, one or more sensor(s) or sensorinterface(s) 610, one or more transceiver(s) 612, one or more optionaldisplay components 614, one or more optionalspeakers(s)/camera(s)/microphone(s) 616, and data storage 620. Theserver 600 may further include one or more bus(es) 618 that functionallycouple various components of the server 600. The server 600 may furtherinclude one or more antenna(e) 630 that may include, without limitation,a cellular antenna for transmitting or receiving signals to/from acellular network infrastructure, a GNSS antenna for receiving GNSSsignals from a GNSS satellite, a Bluetooth antenna for transmitting orreceiving Bluetooth signals, an NFC antenna for transmitting orreceiving NFC signals, and so forth. These various components will bedescribed in more detail hereinafter.

The bus(es) 618 may include at least one of a system bus, a memory bus,an address bus, or a message bus, and may permit the exchange ofinformation (for example, data (including computer-executable code),signaling, etc.) between various components of the server 600. Thebus(es) 618 may include, without limitation, a memory bus or a memorycontroller, a peripheral bus, an accelerated graphics port, and soforth. The bus(es) 618 may be associated with any suitable busarchitecture.

The memory 604 of the server 600 may include volatile memory (memorythat maintains its state when supplied with power) such as RAM and/ornon-volatile memory (memory that maintains its state even when notsupplied with power) such as read-only memory (ROM), flash memory,ferroelectric RAM (FRAM), and so forth. Persistent data storage, as thatterm is used herein, may include non-volatile memory. In certain exampleembodiments, volatile memory may enable faster read/write access thannon-volatile memory. However, in certain other example embodiments,certain types of non-volatile memory (for example, FRAM) may enablefaster read/write access than certain types of volatile memory.

The data storage 620 may include removable storage and/or non-removablestorage including, but not limited to, magnetic storage, optical diskstorage, and/or tape storage. The data storage 620 may providenon-volatile storage of computer-executable instructions and other data.

The data storage 620 may store computer-executable code, instructions,or the like that may be loadable into the memory 604 and executable bythe processor(s) 602 to cause the processor(s) 602 to perform orinitiate various operations. The data storage 620 may additionally storedata that may be copied to the memory 604 for use by the processor(s)602 during the execution of the computer-executable instructions. Morespecifically, the data storage 620 may store one or more operatingsystems (O/S) 622; one or more database management systems (DBMS) 624;and one or more program module(s), applications, engines,computer-executable code, scripts, or the like. Some or all of thesecomponent(s) may be sub-component(s). Any of the components depicted asbeing stored in the data storage 620 may include any combination ofsoftware, firmware, and/or hardware. The software and/or firmware mayinclude computer-executable code, instructions, or the like that may beloaded into the memory 604 for execution by one or more of theprocessor(s) 602. Any of the components depicted as being stored in thedata storage 620 may support functionality described in reference tocorresponding components named earlier in this disclosure.

The processor(s) 602 may be configured to access the memory 604 andexecute the computer-executable instructions loaded therein. Forexample, the processor(s) 602 may be configured to execute thecomputer-executable instructions of the various program module(s),applications, engines, or the like of the server 600 to cause orfacilitate various operations to be performed in accordance with one ormore embodiments of the disclosure. The processor(s) 602 may include anysuitable processing unit capable of accepting data as input, processingthe input data in accordance with stored computer-executableinstructions, and generating output data. The processor(s) 602 mayinclude any type of suitable processing unit.

Referring now to other illustrative components depicted as being storedin the data storage 620, the O/S 622 may be loaded from the data storage620 into the memory 604 and may provide an interface between otherapplication software executing on the server 600 and the hardwareresources of the server 600.

The DBMS 624 may be loaded into the memory 604 and may supportfunctionality for accessing, retrieving, storing, and/or manipulatingdata stored in the memory 604 and/or data stored in the data storage620. The DBMS 624 may use any of a variety of database models (forexample, relational model, object model, etc.) and may support any of avariety of query languages.

Referring now to other illustrative components of the server 600, theinput/output (I/O) interface(s) 606 may facilitate the receipt of inputinformation by the server 600 from one or more I/O devices as well asthe output of information from the server 600 to the one or more I/Odevices. The I/O devices may include any of a variety of components suchas a display or display screen having a touch surface or touchscreen; anaudio output device for producing sound, such as a speaker; an audiocapture device, such as a microphone; an image and/or video capturedevice, such as a camera; a haptic unit; and so forth. The I/Ointerface(s) 606 may also include a connection to one or more of theantenna(e) 630 to connect to one or more networks via a wireless localarea network (WLAN) (such as Wi-Fi) radio, Bluetooth, ZigBee, and/or awireless network radio, such as a radio capable of communication with awireless communication network such as a Long Term Evolution (LTE)network, WiMAX network, 3G network, a ZigBee network, etc.

The server 600 may further include one or more network interface(s) 608via which the server 600 may communicate with any of a variety of othersystems, platforms, networks, devices, and so forth. The networkinterface(s) 608 may enable communication, for example, with one or morewireless routers, one or more host servers, one or more web servers, andthe like via one or more networks.

The sensor(s)/sensor interface(s) 610 may include or may be capable ofinterfacing with any suitable type of sensing device such as, forexample, inertial sensors, force sensors, thermal sensors, photocells,and so forth.

The display component(s) 614 may include one or more display layers,such as LED or LCD layers, touch screen layers, protective layers,and/or other layers. The optional camera(s) of thespeakers(s)/camera(s)/microphone(s) 616 may be any device configured tocapture ambient light or images. The optional microphone(s) of thespeakers(s)/camera(s)/microphone(s) 616 may be any device configured toreceive analog sound input or voice data. The microphone(s) of thespeakers(s)/camera(s)/microphone(s) 616 may include microphones used tocapture sound.

It should be appreciated that the program module(s), applications,computer-executable instructions, code, or the like depicted in FIG. 6as being stored in the data storage 620 are merely illustrative and notexhaustive and that processing described as being supported by anyparticular module may alternatively be distributed across multiplemodule(s) or performed by a different module.

It should further be appreciated that the server 600 may includealternate and/or additional hardware, software, or firmware componentsbeyond those described or depicted without departing from the scope ofthe disclosure.

The user device 650 may include one or more computer processor(s) 652,one or more memory devices 654, and one or more applications, such as avehicle application 656. Other embodiments may include differentcomponents.

The processor(s) 652 may be configured to access the memory 654 andexecute the computer-executable instructions loaded therein. Forexample, the processor(s) 652 may be configured to execute thecomputer-executable instructions of the various program module(s),applications, engines, or the like of the device to cause or facilitatevarious operations to be performed in accordance with one or moreembodiments of the disclosure. The processor(s) 652 may include anysuitable processing unit capable of accepting data as input, processingthe input data in accordance with stored computer-executableinstructions, and generating output data. The processor(s) 652 mayinclude any type of suitable processing unit.

The memory 654 may include volatile memory (memory that maintains itsstate when supplied with power). Persistent data storage, as that termis used herein, may include non-volatile memory. In certain exampleembodiments, volatile memory may enable faster read/write access thannon-volatile memory. However, in certain other example embodiments,certain types of non-volatile memory (for example, FRAM) may enablefaster read/write access than certain types of volatile memory.

Referring now to functionality supported by the user device 650, the AVapplication 656 may be a mobile application executable by the processor652 that can be used to present options and/or receive user inputs ofinformation related to the disclosed embodiments. In addition, the userdevice 650 may communicate with the AV 640 via the network 642 and/or adirect connect, which may be a wireless or wired connection. The userdevice 650 may include a camera, scanner, bio reader or the like tocapture biometric data of a user, perform certain processing steps onthe biometric date, such as extracting features from captured biometricdata, and then communicated those extracted features to one or moreremote servers, such as one or more of cloud-based servers.

It should be appreciated that the program module(s), applications,computer-executable instructions, code, and/or the like depicted in FIG.6 as being stored in the data storage 620 are merely illustrative andnot exhaustive and that processing described as being supported by anyparticular module may alternatively be distributed across multiplemodule(s) or performed by a different module.

NON The autonomous vehicle 640 may include one or more computerprocessor(s) 660, one or more memory devices 662, one or more sensors664, and one or more applications, such as an autonomous drivingapplication 666. Other embodiments may include different components. Acombination or sub combination of these components may be integral tothe controller 606 in FIG. 6.

The processor(s) 660 may be configured to access the memory 662 andexecute the computer-executable instructions loaded therein. Forexample, the processor(s) 660 may be configured to execute thecomputer-executable instructions of the various program module(s),applications, engines, or the like of the device to cause or facilitatevarious operations to be performed in accordance with one or moreembodiments of the disclosure. The processor(s) 660 may include anysuitable processing unit capable of accepting data as input, processingthe input data in accordance with stored computer-executableinstructions, and generating output data. The processor(s) 660 mayinclude any type of suitable processing unit.

The memory 662 may include volatile memory (memory that maintains itsstate when supplied with power) such as random access memory (RAM)and/or non-volatile memory (memory that maintains its state even whennot supplied with power) such as read-only memory (ROM), flash memory,ferroelectric RAM (FRAM), and so forth. Persistent data storage, as thatterm is used herein, may include non-volatile memory. In certain exampleembodiments, volatile memory may enable faster read/write access thannon-volatile memory. However, in certain other example embodiments,certain types of non-volatile memory (e.g., FRAM) may enable fasterread/write access than certain types of volatile memory.

It should further be appreciated that the server 600 may includealternate and/or additional hardware, software, or firmware componentsbeyond those described or depicted without departing from the scope ofthe disclosure.

EXAMPLE EMBODIMENTS

In some instances, the following examples may be implemented together orseparately by the systems and methods described herein.

Example 1 may include a device, comprising: at least one memory devicethat stores computer-executable instructions; and at least one processorconfigured to access the at least one memory device, wherein the atleast one processor is configured to execute the computer-executableinstructions to: determine at least one first sound from environmentalsounds external to a cabin of a vehicle, wherein the cabin is configuredto reduce a volume of the environmental sounds below a threshold;determine location information comprising a direction and a distance ofthe first sound with respect to the vehicle; generate a second soundbased on the first sound and the location information that reproduces aspectral feature of the first sound; and cause at least a portion of thesecond sound to play on a speaker of the cabin.

Example 2 may include the device of example 1 and/or some other exampleherein, wherein generating the second sound comprises reducing noiseassociated with the first sound by an amount that is based on a userpreference.

Example 3 may include the device of example 1 and/or some other exampleherein, wherein determining the first sound comprises: determining aplurality of sounds and respective sound types from the environmentalsounds; and filtering out sounds having predetermined sound types fromthe environmental sounds.

Example 4 may include the device of example 3 and/or some other exampleherein, wherein determining the sound types comprises assigningrespective priorities to the sounds, and wherein filtering out soundsfurther comprises filtering out sounds that have priorities belowrespective thresholds.

Example 5 may include the device of example 1 and/or some other exampleherein, wherein determining the location information comprisesperforming acoustic trilateration using vehicle-to-vehicle (V2V)communications with other vehicles configured to detect theenvironmental sounds.

Example 6 may include the device of example 1 and/or some other exampleherein, wherein determining the location information comprisesdetermining a Doppler shift associated with the first sound.

Example 7 may include the device of example 1 and/or some other exampleherein, wherein causing to play at least the portion of the second soundcomprises projecting the second sound such that the second sound has aperceived location that is similar to the first sound.

Example 8 may include the device of example 1 and/or some other exampleherein, further comprising causing to present an image based on thesecond sound on a display associated with the cabin.

Example 9 may include the device of example 1 and/or some other exampleherein, wherein the computer-executable instructions further comprisecomputer-executable instructions to: obtain at least one image of anenvironment external to the cabin; determine an emergency conditionbased on an analysis of the image; cause to stop playing the portion ofthe second sound; and cause to play a third sound on the speaker basedon the image.

Example 10 may include a method, comprising: detecting environmentalsounds external to a cabin of a vehicle and determining at least onefirst sound from the environmental sounds, wherein the cabin isconfigured to reduce a volume of the environmental sounds below athreshold; determining location information comprising a direction and adistance of the first sound with respect to the vehicle; and generatinga second sound based on the first sound and the location informationthat reproduces a spectral feature of the first sound.

Example 11 may include the method of example 10 and/or some otherexample herein, wherein generating the second sound comprises reducingnoise associated with the first sound by an amount that is based on auser preference.

Example 12 may include the method of example 10, and/or some otherexample herein wherein determining the first sound comprises:determining a plurality of sounds and respective sound types from theenvironmental sounds; and filtering out sounds having predeterminedsound types from the environmental sounds.

Example 13 may include the method of example 10 and/or some otherexample herein, wherein determining the location information comprisesperforming acoustic trilateration using vehicle-to-vehicle (V2V)communications with other vehicles configured to detect theenvironmental sounds.

Example 14 may include the method of example 10 and/or some otherexample herein, further comprising causing to present an image based onthe second sound on a display associated with the cabin.

Example 15 may include the method of example 10 and/or some otherexample herein, further comprising: obtaining at least one image of anenvironment external to the cabin; determining a condition based on ananalysis of the image; causing to stop playing the portion of the secondsound; and causing to play a third sound on the speaker based on theimage.

Example 16 may include a non-transitory computer-readable medium storingcomputer-executable instructions which, when executed by a processor,cause the processor to perform operations comprising: determining atleast one first sound from environmental sounds external to a cabin of avehicle, wherein the cabin is configured to reduce a volume of theenvironmental sounds below a threshold; determining location informationcomprising a direction and a distance of the first sound with respect tothe vehicle; and generating a second sound based on the first sound andthe location information that reproduces a spectral feature of the firstsound.

Example 17 may include the non-transitory computer-readable medium ofexample 16 and/or some other example herein, wherein generating thesecond sound comprises reducing noise associated with the first sound byan amount that is based on a user preference.

Example 18 may include the non-transitory computer-readable medium ofexample 16 and/or some other example herein, wherein determining thefirst sound comprises: determining a plurality of sounds and respectivesound types from the environmental sounds; and filtering out soundshaving predetermined sound types from the environmental sounds.

Example 19 may include the non-transitory computer-readable medium ofexample 16 and/or some other example herein, wherein determining thelocation information comprises performing acoustic trilateration usingvehicle-to-vehicle (V2V) communications with other vehicles configuredto detect the environmental sounds.

Example 20 may include the non-transitory computer-readable medium ofexample 16 and/or some other example herein, further comprisingcomputer-readable instructions to: obtain at least one image of anenvironment external to the cabin; determine a condition based on ananalysis of the image; cause to stop playing the portion of the secondsound; and cause to play a third sound on the speaker based on theimage.

Although specific embodiments of the disclosure have been described, oneof ordinary skill in the art will recognize that numerous othermodifications and alternative embodiments are within the scope of thedisclosure. For example, any of the functionality and/or processingcapabilities described with respect to a particular device or componentmay be performed by any other device or component. Further, whilevarious illustrative implementations and architectures have beendescribed in accordance with embodiments of the disclosure, one ofordinary skill in the art will appreciate that numerous othermodifications to the illustrative implementations and architecturesdescribed herein are also within the scope of this disclosure.

Blocks of the block diagrams and flow diagrams support combinations ofmeans for performing the specified functions, combinations of elementsor steps for performing the specified functions, and program instructionmeans for performing the specified functions. It will also be understoodthat each block of the block diagrams and flow diagrams, andcombinations of blocks in the block diagrams and flow diagrams, may beimplemented by special-purpose, hardware-based computer systems thatperform the specified functions, elements or steps, or combinations ofspecial-purpose hardware and computer instructions.

A software component may be coded in any of a variety of programminglanguages. An illustrative programming language may be a lower-levelprogramming language such as an assembly language associated with aparticular hardware architecture and/or operating system platform. Asoftware component comprising assembly language instructions may requireconversion into executable machine code by an assembler prior toexecution by the hardware architecture and/or platform.

A software component may be stored as a file or other data storageconstruct. Software components of a similar type or functionally relatedmay be stored together such as, for example, in a particular directory,folder, or library. Software components may be static (for example,pre-established or fixed) or dynamic (for example, created or modifiedat the time of execution).

Software components may invoke or be invoked by other softwarecomponents through any of a wide variety of mechanisms. Invoked orinvoking software components may comprise other custom-developedapplication software, operating system functionality (for example,device drivers, data storage (for example, file management) routines,other common routines and services, etc.), or third-party softwarecomponents (for example, middleware, encryption, or other securitysoftware, database management software, file transfer or other networkcommunication software, mathematical or statistical software, imageprocessing software, and format translation software).

Software components associated with a particular solution or system mayreside and be executed on a single platform or may be distributed acrossmultiple platforms. The multiple platforms may be associated with morethan one hardware vendor, underlying chip technology, or operatingsystem. Furthermore, software components associated with a particularsolution or system may be initially written in one or more programminglanguages but may invoke software components written in anotherprogramming language.

Computer-executable program instructions may be loaded onto aspecial-purpose computer or other particular machine, a processor, orother programmable data processing apparatus to produce a particularmachine, such that execution of the instructions on the computer,processor, or other programmable data processing apparatus causes one ormore functions or operations specified in the flow diagrams to beperformed. These computer program instructions may also be stored in acomputer-readable storage medium (CRSM) that upon execution may direct acomputer or other programmable data processing apparatus to function ina particular manner, such that the instructions stored in thecomputer-readable storage medium produce an article of manufactureincluding instruction means that implement one or more functions oroperations specified in the flow diagrams. The computer programinstructions may also be loaded onto a computer or other programmabledata processing apparatus to cause a series of operational elements orsteps to be performed on the computer or other programmable apparatus toproduce a computer-implemented process.

Although embodiments have been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the disclosure is not necessarily limited to the specific featuresor acts described. Rather, the specific features and acts are disclosedas illustrative forms of implementing the embodiments. Conditionallanguage, such as, among others, “can,” “could,” “might,” or “may,”unless specifically stated otherwise, or otherwise understood within thecontext as used, is generally intended to convey that certainembodiments could include, while other embodiments do not include,certain features, elements, and/or steps. Thus, such conditionallanguage is not generally intended to imply that features, elements,and/or steps are in any way required for one or more embodiments or thatone or more embodiments necessarily include logic for deciding, with orwithout user input or prompting, whether these features, elements,and/or steps are included or are to be performed in any particularembodiment.

What is claimed is:
 1. A device, comprising: at least one memory devicethat stores computer-executable instructions; and at least one processorconfigured to access the at least one memory device, wherein the atleast one processor is configured to execute the computer-executableinstructions to: determine at least one first sound from environmentalsounds external to a cabin of a vehicle, wherein the cabin is configuredto reduce a volume of the environmental sounds below a threshold;determine location information comprising a direction and a distance ofthe first sound with respect to the vehicle; generate a second soundbased on the first sound and the location information that reproduces aspectral feature of the first sound; and cause at least a portion of thesecond sound to play on a speaker of the cabin, wherein causing to playat least the portion of the second sound comprises projecting the secondsound such that the second sound has a perceived location that issimilar to the first sound.
 2. The device of claim 1, wherein generatingthe second sound comprises reducing noise associated with the firstsound by an amount that is based on a user preference.
 3. The device ofclaim 1, wherein determining the first sound comprises: determining aplurality of sounds and respective sound types from the environmentalsounds; and filtering out sounds having predetermined sound types fromthe environmental sounds.
 4. The device of claim 3, wherein determiningthe sound types comprises assigning respective priorities to the sounds,and wherein filtering out sounds further comprises filtering out soundsthat have priorities below respective thresholds.
 5. The device of claim1, wherein determining the location information comprises performingacoustic trilateration using vehicle-to-vehicle (V2V) communicationswith other vehicles configured to detect the environmental sounds. 6.The device of claim 1, wherein determining the location informationcomprises determining a Doppler shift associated with the first sound.7. The device of claim 1, further comprising causing to present an imagebased on the second sound on a display associated with the cabin.
 8. Thedevice of claim 1, wherein the computer-executable instructions furthercomprise computer-executable instructions to: obtain at least one imageof an environment external to the cabin; determine an emergencycondition based on an analysis of the image; cause to stop playing theportion of the second sound; and cause to play a third sound on thespeaker based on the image.
 9. A method, comprising: detectingenvironmental sounds external to a cabin of a vehicle and determining atleast one first sound from the environmental sounds, wherein the cabinis configured to reduce a volume of the environmental sounds below athreshold; determining location information comprising a direction and adistance of the first sound with respect to the vehicle; and generatinga second sound based on the first sound and the location informationthat reproduces a spectral feature of the first sound, whereindetermining the location information comprises performing acoustictrilateration using vehicle-to-vehicle (V2V) communications with othervehicles configured to detect the environmental sounds.
 10. The methodof claim 9, wherein generating the second sound comprises reducing noiseassociated with the first sound by an amount that is based on a userpreference.
 11. The method of claim 9, wherein determining the firstsound comprises: determining a plurality of sounds and respective soundtypes from the environmental sounds; and filtering out sounds havingpredetermined sound types from the environmental sounds.
 12. The methodof claim 9, further comprising causing to present an image based on thesecond sound on a display associated with the cabin.
 13. The method ofclaim 9, further comprising: obtaining at least one image of anenvironment external to the cabin; determining a condition based on ananalysis of the image; causing to stop playing the second sound; andcausing to play a third sound on a speaker based on the image.
 14. Anon-transitory computer-readable medium storing computer-executableinstructions which, when executed by a processor, cause the processor toperform operations comprising: determining at least one first sound fromenvironmental sounds external to a cabin of a vehicle, wherein the cabinis configured to reduce a volume of the environmental sounds below athreshold; determining location information comprising a direction and adistance of the first sound with respect to the vehicle; generating asecond sound based on the first sound and the location information thatreproduces a spectral feature of the first sound; obtaining at least oneimage of an environment external to the cabin; determining a conditionbased on an analysis of the image; causing to stop playing at least aportion of the second sound; and causing to play a third sound on aspeaker based on the image.
 15. The non-transitory computer-readablemedium of claim 14, wherein generating the second sound comprisesreducing noise associated with the first sound by an amount that isbased on a user preference.
 16. The non-transitory computer-readablemedium of claim 14, wherein determining the first sound comprises:determining a plurality of sounds and respective sound types from theenvironmental sounds; and filtering out sounds having predeterminedsound types from the environmental sounds.
 17. The non-transitorycomputer-readable medium of claim 14, wherein determining the locationinformation comprises performing acoustic trilateration usingvehicle-to-vehicle (V2V) communications with other vehicles configuredto detect the environmental sounds.
 18. A device, comprising: at leastone memory device that stores computer-executable instructions; and atleast one processor configured to access the at least one memory device,wherein the at least one processor is configured to execute thecomputer-executable instructions to: determine at least one first soundfrom environmental sounds external to a cabin of a vehicle, wherein thecabin is configured to reduce a volume of the environmental sounds belowa threshold; determine location information comprising a direction and adistance of the first sound with respect to the vehicle; generate asecond sound based on the first sound and the location information thatreproduces a spectral feature of the first sound; and cause at least aportion of the second sound to play on a speaker of the cabin, whereindetermining the location information comprises determining a Dopplershift associated with the first sound.
 19. A device, comprising: atleast one memory device that stores computer-executable instructions;and at least one processor configured to access the at least one memorydevice, wherein the at least one processor is configured to execute thecomputer-executable instructions to: determine at least one first soundfrom environmental sounds external to a cabin of a vehicle, wherein thecabin is configured to reduce a volume of the environmental sounds belowa threshold, wherein determining the first sound comprises: determininga plurality of sounds and respective sound types from the environmentalsounds, wherein determining the sound types comprises assigningrespective priorities to the sounds; and filtering out sounds havingpredetermined sound types from the environmental sounds, whereinfiltering out sounds comprises filtering out sounds that have prioritiesbelow respective thresholds; determine location information comprising adirection and a distance of the first sound with respect to the vehicle;generate a second sound based on the first sound and the locationinformation that reproduces a spectral feature of the first sound; andcause at least a portion of the second sound to play on a speaker of thecabin.